For software verification and validation at the system level, a set of stimuli and expected reactions of the system are traditionally specified. The stimuli and expected reactions are derived directly from corresponding requirements. The scope of these requirements, and therefore also the scope of the test cases, are generally restricted to a finite set of identified application cases. Typically, the test cases are specified by test engineers by using software test tools. Such tools may comprise functions for assisting test automation and for increasing the test maturity (version management, task tracker, graphical test specification, etc.).
There are also formal verification methods for more systematic checks that software implementations do not infringe particular rules.
The testing of software for autonomous vehicles at the system level entails the new challenge that the set of application cases belonging to such a system is almost infinite. In fact, a system must thus be capable of dealing with most driving situations with which any driver is possibly confronted during his life. Different drivers are faced with different types of application cases, depending on a large number of environmental factors (other road users, time of day, weather, health, vehicle wear, road condition, etc.). There therefore seems to be an extremely large number of test combinations.
Virtual test environments for autonomous vehicles are known. For instance, the publication “Vehicle in the Loop” in Journal “Elektrointegration”, ATZ 01/2008 vol. 110, pp. 2-8, describes a test and simulation environment for driver assistance systems, in which a real test vehicle, which moves not in public road traffic but on a free surface or a testing area, is combined with a driving simulator. With such a test setup, referred to as a “Vehicle in the Loop”, it is possible to test without risk how driver assistance functions react to other virtual traffic or other virtual objects in a virtual driving environment.
The publication “Towards a Hybrid Real/Virtual Simulation of Autonomous Vehicles for Critical Scenarios”, SIMUL 2014, ISBN 978 1 61208 371 1, pp. 14-17, describes hybrid real/virtual simulations of autonomous vehicles for critical situations, specifically with virtual sensors and a real vehicle as “Hardware in the simulation loop”.
There is, however, still a need for improved tools and methods which can generate test cases for autonomous vehicles more efficiently.